There is currently a proliferation of organizational networked computing systems. Every type of organization, be it a commercial company, a university, a bank, a government agency or a hospital, heavily relies on one or more networks interconnecting multiple computing nodes. Failures of the networked computing system of an organization, or even of only a portion of it, might cause significant damage, up to completely shutting down all operations. Additionally, much of the data of the organization, if not all the data, exist somewhere on its networked computing system, including all confidential data comprising the “crown jewels” of the organization, such as prices, details of customers, purchase orders, employees' salaries, technical formulas, etc. Loss of such data or leaks of such data to unauthorized external entities might be disastrous for the organization.
Many organizational networks are connected to the Internet at least through one network node, and consequently may be subject to attacks by computer hackers or by hostile adversaries. Quite often the newspapers report incidents in which websites crashed, sensitive data was stolen, or service to customers was denied, where the failures were the results of hostile penetration into an organization's networked computing system.
Thus, many organizations invest a lot of efforts and costs in preventive means designed to protect their computing networks against potential threats. There are many defensive products offered in the market claiming to provide protection against one or more known modes of attack, and many organizations arm themselves to the teeth with multiple products of this kind.
However, it is difficult to tell how effective such products really are in achieving their stated goals of blocking hostile attacks, and consequently most CISOs (Computer Information Security Officers) will admit (maybe only off the record), that they don't really know how well they can withstand an attack from a given adversary. The only way to really know the strength and security of a system, is by trying to attack it as a real adversary would. This is known as red-teaming or penetration testing (pen testing, in short), and is a very common approach that is even required by regulation in some developed countries.
Penetration testing requires highly talented people to man the testing team. Those people should be familiar with each and every publicly known vulnerability and attacking method and should also have a very good familiarity with networking techniques and multiple operating systems implementations. Such people are hard to find and therefore many organizations give up establishing their own penetration testing teams and resort to hiring external expert consultants for carrying out that role (or completely give up penetration testing). However, external consultants are expensive and therefore are typically called in only for brief periods separated by long intervals in which no penetration testing is carried out. This makes the penetration testing ineffective, as vulnerabilities caused by new attacks, that appear almost daily, are discovered only months after becoming serious threats to the organization.
Additionally, even rich organizations that can afford hiring talented experts for in-house penetration testing teams do not achieve good protection. Testing for vulnerabilities of a large network containing many types of computers, operating systems, network routers and other devices is both a very complex and a very tedious process. The process is prone to human errors such as missing testing for certain threats or misinterpreting the damages of certain attacks. Additionally, because a process of full testing against all threats is quite long, the organization might again end with a too long discovery period after a new threat appears.
In view of the above difficulties, several vendors offer automated penetration testing systems. Such systems automatically discover and report vulnerabilities of a networked system, potential damages that might be caused to the networked system, and potential trajectories of attack that may be employed by an attacker.
A penetration testing process involves at least the following main functions: (i) a reconnaissance function, (ii) an attack function, and (iii) a reporting function. The process may also include additional functions, for example a cleanup function that restores the tested networked system to its original state as it was before the test. In an automated penetration testing system, at least one of the above three functions is at least partially automated, and typically two or three of them are at least partially automated.
A reconnaissance function is the function within a penetration testing system that handles the collection of data about the tested networked system. The collected data may include internal data of networks nodes, data about network traffic within the tested networked system, business intelligence data of the organization owning the tested networked system, etc. The functionality of a prior art reconnaissance function can be implemented, for example, by software executing in a server that is not one of the network nodes of the tested networked system, where the server probes the tested networked system for the purpose of collecting data about it.
An attack function is the function within a penetration testing system that handles the determination of whether security vulnerabilities exist in the tested networked system based on data collected by the reconnaissance function. The functionality of a prior art attack function can be implemented, for example, by software executing in a server that is not one of the nodes of the tested networked system, where the server attempts to attack the tested networked system for the purpose of verifying that it can be compromised.
A reporting function is the function within a penetration testing system that handles the reporting of results of the penetration testing system. The functionality of a prior art reporting function may be implemented, for example, by software executing in the same server that executes the functionality of the attack function, where the server reports the findings of the attack function to an administrator or a CISO of the tested networked system.
FIG. 1A (PRIOR ART) is a block diagram of code modules of a typical penetration testing system. FIG. 1B (PRIOR ART) is a related flow-chart.
In FIG. 1A, code for the reconnaissance function, for the attack function, and for the reporting function are respectively labelled as 20, 30 and 40, and are each schematically illustrated as part of a penetration testing system code module (PTSCM) labelled as 10. The term ‘code’ is intended broadly and may include any combination of computer-executable code and computer-readable data which when read affects the output of execution of the code. The computer-executable code may be provided as any combination of human-readable code (e.g. in a scripting language such as Python), machine language code, assembler code and byte code, or in any form known in the art. Furthermore, the executable code may include any stored data (e.g. structured data) such as configuration files, XML files, and data residing in any type of database (e.g. a relational database, an object-database, etc.).
In one example and as shown in FIG. 1B, the reconnaissance function (performed in step S21 by execution of reconnaissance function code 20), the attack function (performed in step S31 by execution of attack function code 30) and the reporting function (performed in step S41 by execution of reporting function code 40) are executed in strictly sequential order so that first the reconnaissance function is performed by executing code 20 thereof, then the attack function is performed by executing code 30 thereof, and finally the reporting function is performed 40 by executing code thereof.
However, the skilled artisan will appreciate that this order is just one example, and is not a requirement. For example, the attack and the reporting functions may be performed in parallel or in an interleaved way, with the reporting function reporting first results obtained by the attack function, while the attack function is working on additional results.
Similarly, the reconnaissance and the attack functions may operate in parallel or in an interleaved way, with the attack function detecting a vulnerability based on first data collected by the reconnaissance function, while the reconnaissance function is working on collecting additional data.
FIG. 1A also illustrates code of an optional cleanup function which is labeled as 50. Also illustrated in FIG. 1B is step S51 of performing a cleanup function—e.g. by cleanup function code 50 of FIG. 1A.
“A campaign of penetration testing” is a specific run of a specific test of a specific networked system by the penetration testing system.
A penetration-testing-campaign module may comprise at least part of reconnaissance function code 20, attack function code 30, reporting function code 40 and optionally cleanup function code 50—for example, in combination with suitable hardware (e.g. one or more computing device(s) 110 and one or more processor(s) 120 thereof, see FIG. 2) for executing the code.
FIG. 2 illustrates a prior art computing device 110 which may have any form-factor including but not limited to a laptop, a desktop, a mobile phone, a server, a tablet, or any other form factor. The computing device 110 in FIG. 2 includes (i) computer memory 160 which may store code 180; (ii) one or more processors 120 (e.g. central-processing-unit (CPU)) for executing code 180; (iii) one or more human-interface device(s) 140 (e.g. mouse, keyboard, touchscreen, gesture-detecting apparatus including a camera, etc.) or an interface (e.g. USB interface) to receive input from a human-interface device; (iv) a display device 130 (e.g. computer screen) or an interface (e.g. HDMI interface, USB interface) for exporting video to a display device and (v) a network interface 150 (e.g. a network card, or a wireless modem).
Memory 160 may include any combination of volatile (e.g. RAM) and non-volatile (e.g. ROM, flash, disk-drive) memory. Code 180 may include operating-system code—e.g. Windows®, Linux®, Android®, Mac-OS®.
Computing device 110 may include a user-interface for receiving input from a user (e.g. manual input, visual input, audio input, or input in any other form) and for visually displaying output. The user-interface (e.g. graphical user interface (GUI)) of computing device 110 may thus include the combination of HID device 140 or an interface thereof (i.e. in communication with an external HID device), display device 130 or an interface thereof (i.e. in communication with an external display device), and user-interface (UI) code stored in memory 160 and executed by one or more processor(s) 120. The user-interface may include one or more GUI widgets such as labels, buttons (e.g. radio buttons or check boxes), sliders, spinners, icons, windows, panels, text boxes, and the like.
In one example, a penetration testing system is the combination of (i) code 10 (e.g. including reconnaissance function code 20, attack function code 30, reporting function code 40, and optionally cleaning function code 50); and (ii) one or more computing devices 110 which execute the code 10. For example, a first computing device may execute a first portion of code 10 and a second computing device (e.g. in networked communication with the first computing device) may execute a second portion of code 10.
Penetration testing systems may employ different types of architectures, each having its advantages and disadvantages. Examples are actual attack penetration testing systems, simulated penetration testing systems and reconnaissance agent penetration testing systems. See the Definitions section for more details about these types of penetration testing systems.
The Problem to Solve
As described above, an automated penetration testing system carries out a penetration testing campaign in order to test how well the tested networked system is defended against cyber-attacks. Each penetration testing campaign has an associated “goal of an attacker”. The “goal of an attacker” of a campaign indicates what the attacker of the campaign is attempting to achieve by attacking the networked system being tested. In other words, what is the criterion according to which the attack will be considered a success or a failure and/or to what extent was the attack successful. Examples of goals of attackers include: exporting a specific file out of the networked system, shutting down a specific network node of the networked system, encrypting five Excel files in a specific node of the networked system (for example as part of a demand for ransom), and the like.
If the penetration testing campaign succeeds in achieving the goal, an attacker might be able to do the same, and therefore the networked system is vulnerable to attack.
Typically, the method for an attacker to achieve the goal comprises an ordered series of steps the attacker would take in order to achieve the goal. An exemplary method for achieving a goal of controlling network node A (e.g. the CEO's computer), illustrated in FIG. 3A, may include:    1. Using method Z (e.g. using a known weakness in Microsoft Windows's networking mechanism) (step 300), attacking and compromising network node B (e.g. the management floor administrator's computer) (sub-goal 302).    2. Once network node B is under the attacker's control, using method Y (e.g. using a known method of extracting password hash codes files) (step 304) for extracting from network node B a file containing password hash codes of the management floor network nodes (sub-goal 306).    3. Based on the obtained password hash codes of the management floor network nodes and using method X (e.g. using a known method for recovering passwords from their hash codes) (step 308), recovering passwords of the management floor network nodes corresponding to the obtained password hash codes (sub-goal 310). It is assumed the CEO's password is kept separately and is not obtained in this step.    4. Based on the recovered passwords of the management floor network nodes (which include the password of the CEO's personal assistant's computer), and using method W (e.g. logging into a computer using its known password) (step 312), attacking and compromising network node C (e.g. the CEO's personal assistant's computer) (sub-goal 314).    5. Once network node C (e.g. the CEO's personal assistant's computer) is under the attacker's control, using method V (e.g. using a known weakness in Microsoft Windows's shared folder mechanism) (step 316) for attacking and compromising network node A (e.g. the CEO's computer) (goal 318).
As can be seen in FIG. 3A, a way for accomplishing the goal of the attacker, also termed as “a path of attack”, includes a sequence of attacker steps, each attacker step moving the attacker from one sub-goal to a subsequent sub-goal.
For example, attacker step 304 moves the attacker from sub-goal 302 of “controlling network node B” to sub-goal 306 of “having the password hash codes of the management floor network nodes”, and attacker step 308 moves the attacker from sub-goal 306 of “having the password hash codes of the management floor network nodes” to sub-goal 310 of “having the passwords of the management floor network nodes”. The first attacker step 302 moves the attacker from the starting state, which is a dummy sub-goal that is assumed to be satisfied when starting the penetration test, and the last attacker step 316 moves the attacker to sub-goal 318 that is identical to the true goal of the attacker of the penetration testing campaign.
Note that FIG. 3A represents a path of attack of a networked system as a directed graph in which sub-goals and attacker steps are represented by nodes of the graph. FIG. 3B shows an alternative representation of the exemplary method shown in FIG. 3A, in which a path of attack of a networked system is represented as a graph in which sub-goals are represented by nodes of the graph, while attacker steps are represented by directed edges connecting sub-goals. In the graph of FIG. 3B, the steps and sub-goals are given the same reference numerals as in FIG. 3A. Other forms of representing paths of attack of a networked system are also possible.
For example, paths of attack of a networked system may be represented by a list, where each member of the list corresponds to one path of attack of the networked system. Each member of the list, or each such path of attack, is itself represented as a list, including items corresponding to all attacker steps and to all sub-goals included in that path of attack. FIG. 3C illustrates an example of such representation, showing a list 320 including two paths of attack 322 and 324.
The representation methods of FIGS. 3A, 3B, and 3C are equivalent, and any one of them, or any other equivalent representation method, may be used for implementing the methods and systems of the present disclosure. The use of any specific representation method in any figure or any explanation of the present disclosure should not be construed as limiting the scope of the claimed invention to that specific representation method.
Typically, a description of the method for the attacker to achieve the goal, which method was discovered by a penetration test, is displayed to the user of the penetration testing system, and optionally also mailed to other people such as the administrator or CISO of the organization owning the tested networked system.
Most penetration testing systems also provide to the user a recommendation as to how to block the discovered method of attack. In the above example, the recommendation may be to upgrade the Microsoft Windows OS installed on network node B to a later version that does not have the weakness used by step 300. Alternatively, the recommendation may be to have the CEO's personal assistant computer, network node C, use a two-factor authentication method, so that knowledge of the password for network node C is insufficient for logging into this computer, thus blocking step 312. Alternatively, the recommendation may be to disable the folder sharing mechanism that is used in step 316 in the CEO's computer, network node A, and ask the CEO and his/her personal assistant to share files using other mechanisms. In the above example, each step in the discovered method to attack results in a corresponding recommendation for a remediation action that is aimed at blocking that step.
Since methods for attack might be complex and may include dozens of attacker steps, sometimes having multiple alternatives for some of those attacker steps, choosing the best recommendation to provide to the user for blocking the attack might become quite difficult. The prior art discloses many approaches, some of which are:    1. Using cost of exploitation as the determining consideration. The cost of exploitation of an attacker step is a measure of how difficult it is for an attacker to carry out that attacker step. For example, the attacker step known as “ARP Spoofing” is costlier for the attacker than an attacker step that uses a publicly available exploit kit. The cost of exploitation may be represented by a numeric score within a given range, typically (but not necessarily) with a higher score representing a costlier method. For example, the given range may be [0 . . . 10], with a cost of exploitation of ARP Spoofing being 7, and a cost of exploitation of any attacker step using a publicly available exploit kit being 2. A penetration testing system using this approach may recommend to the user to invest in blocking the attacker step having the lowest cost of exploitation. It is noted that in the example presented above, which includes a single and linear path of attack, it does not matter which attacker step of the path of attack is blocked, as blocking any attacker step protects the tested networked system from the discovered method of attack. However, when dealing with more complex cases, the advantage of selecting the attacker step to block based on cost of exploitation becomes clear, as will be evident hereinbelow.    2. Using cost of remediation as the determining consideration. The cost of remediation of an attacker step is a measure of how expensive it is for the organization owning the tested networked system to block that attacker step. For example, an attacker step that can be blocked by simply installing a security patch for a software application (e.g. Microsoft Word) is much less costly to fix than an attacker step that requires buying and installing a new router in order to split an existing sub-network into two different sub-networks.            The cost of remediation may be represented by a numeric score within a given range, typically (but not necessarily) with a higher score representing a costlier method. For example, the given range may be [0 . . . 10], with a cost of remediation requiring only installing a patch being 1, and a cost of remediation requiring a new router being 8. A penetration testing system using this approach may recommend to the user to invest in blocking the attacker step having the lowest cost of remediation, thus blocking an attacker from achieving its goal while investing the smallest budget.            3. Using probability of success as the determining consideration. The probability of success of an attacker step is a measure of how probable is it that execution of the attacker step by the attacker will succeed in achieving the sub-goal that the attacker step is intended to achieve, taking into account the currently available knowledge regarding the state of the attacked networked system. For example, an attacker step that is based on exploiting a known Windows 7 vulnerability may have high probability of success when applied to a network node having the original version of the OS installed, and may have a low probability of success when applied to a network node in which a certain security patch has been installed.            Typically, probabilities of success are expressed in percentages in the range of 0% to 100%. Alternatively, the probabilities of success may be represented by numeric values in the range of zero to one, where zero corresponds to 0% and one corresponds to 100%. However, any other numerical scale may be used for representing probabilities of success, provided that the scale is a monotonically increasing or monotonically decreasing function of how probable is it that the attacker step will succeed in achieving its sub-goal.        A penetration testing system using this approach may recommend to the user to invest in blocking the attacker step having the highest probability of success.            4. Using a combination of any of the previous considerations. For example, a combination of cost of exploitation and cost of remediation may be used as the determining considerations. A function of both factors is defined, and the recommendation of which attacker step to block is selected based on values of the function for the various attacker steps of the path of attack.            For example, the value of the function for each attacker step may be defined to be the multiplication of the cost of exploitation of the attacker step by the cost of remediation of the attacker step, and the recommendation may be to block the attacker step having the lowest value of the function.        In another example, the value of the function for each attacker step may be defined as follows—if the cost of exploitation of the attacker step is lower than five, then the value of the function is the cost of remediation multiplied by three, and if the cost of exploitation of the attacker step is equal to or greater than five, then the value of the function is the cost of remediation multiplied by seven. Again, the recommendation may be to block the attacker step having the lowest value of the function.        
The example of a method for achieving an attacker's goal of compromising a networked system illustrated in FIGS. 3A and 3B has a relatively simple attacker's goal and a single and relatively simple path of attack for the attacker to achieve his goal. Consequently, deciding which recommendation to provide to the user is also a relatively simple task.
In other cases, the attacker's goal and the options available for the attacker to achieve that goal may be more complex, and consequently the selection of a suitable recommendation to provide to the user is also more complex. For example, the attacker's goal may be to control either one of the CEO's computer and the CFO's computer. In this case, the method to achieve the goal contains two separate “branches” (i.e. paths of attack), merging in the final sub-goal (see FIG. 4, which is equivalent to FIG. 3C, but is presented as a graph rather than as a list).
One branch of the graph of FIG. 4 is identical to the graph illustrated in FIG. 3A, and steps and sub-goals thereof are indicated by the same reference numerals, while the second branch includes the following steps and sub-goals:    1. Using method U (e.g. using a known weakness in Microsoft Windows's RPC mechanism) (step 400), attacking and compromising network node D (e.g. the finance floor administrator's computer) (sub-goal 402).    2. Once network node D is under the attacker's control, using method T (e.g. using a known method of extracting password files) (step 404) for extracting from network node D a file containing passwords of the finance floor network nodes (sub-goal 406). It is assumed the CFO's password is kept separately and is not obtained in this step.    3. Based on the obtained passwords of the finance floor network nodes (which include the password of the CFO's personal assistant's computer), and using method S (e.g. logging into a computer using its known password) (step 408), attacking and compromising network node E (e.g. the CFO's personal assistant's computer) (sub-goal 410).    4. Once network node E (the CFO's personal assistant's computer) is under the attacker's control, using method R (e.g. using a known weakness in Microsoft Windows's file sharing mechanism) (step 412) for attacking and compromising network node F (e.g. the CFO's computer) (sub-goal 414).
It is clear that in this case it is not possible to block all possibilities of attack by blocking a single attacker step. Each of the two branches has to be separately blocked, requiring blocking of at least one attacker step in each branch.
A possible algorithm for selecting recommendations of steps to be blocked in this case may operate as follows:    A. Determine the best attacker step to block in a first branch of the two, assuming the other branch does not exist.    B. Determine the best attacker step to block in the other branch, assuming the first branch does not exist.    C. Recommend blocking both of the steps determined in A and B.
Blocking one attacker step in each of the two branches guarantees that both branches become unusable (or at least less usable) for the attacker. The step recommended for blocking the first branch and the step recommended for blocking the second branch may be determined using any method known in the art—based on cost of exploitation, based on cost of remediation, based on both cost of exploitation and cost of remediation, etc. It should be noted that the determination for each of the branches is completely independent of the determination for the other branch. Moreover, the method of determination may be different for the two branches—for example, in one branch the determination may be based on cost of remediation, while in the other branch the determination may be based on a combination of cost of remediation and probability of success.
Blocking one attacker step in each path of attack is useful when the owner of the tested networked system blocks as many attacker steps as are required for completely blocking all possibilities for the attackers to compromise the networked system. However, this is not always the case in the real world. Remediation actions recommended for blocking all possibilities of attack might be highly expensive and their combined costs might exceed the budget available for protecting the networked system. For example, blocking a certain attacker step may require installing a certain defensive application on each node of the networked system. Even for an inexpensive application costing just $10 per node, in a large network of 10,000 network nodes this translates to a direct cost of $100,000, prior to training of users and other indirect costs.
Therefore, some penetration testing systems provide their recommendations as a list, ordered by priority. This way the available funds can be allocated to implementing the recommendations at the top of the list, which are the most urgent and/or important to fix. Even though such partial implementation of the recommendations list might not provide a fool-proof solution, leaving some unblocked attack paths available to potential attackers, it nevertheless makes the best use of the available funds and improves the security posture of the networked system to the maximal possible under the existing budget constraints.
Algorithms for selecting the highest priority recommendation may operate in the following high-level way:                1. Determine the list of “branches” (i.e. paths of attack) available for a potential attacker to achieve his goal.        2. If there is only one branch, define it to be the “critical branch”.        3. Otherwise, if there are multiple branches, evaluate each of the branches according to some criteria and generate a numerical score for each branch. Based on generated numerical scores for all the branches, define one of the branches to be the “critical branch”.        4. Determine, based on some criteria, one of the attacker steps of the critical branch to be the best recommendation.        
The following is an example of such an algorithm:                1. Determine the list of “branches” (i.e. paths of attack) available for a potential attacker to achieve his goal.        2. If there is only one branch, define it to be the “critical branch”.        3. Otherwise, if there are multiple branches:                    a. For each branch, calculate a “cost of exploitation” score. The cost of exploitation score of a branch is the sum of the costs of exploitation of all the attacker steps included in the branch.            b. The branch having the lowest cost of exploitation score of all branches is defined to be the “critical branch”.                        4. Determine which attacker step of the critical branch has the lowest cost of remediation. This step is recommended as the highest priority attacker step to be blocked.        
FIG. 5 demonstrates how the exemplary algorithm described above may be applied to the method of attack illustrated in FIG. 4. In FIG. 5, the cost of exploitation and cost of remediation of each attacker step are provided. The format used in FIG. 5 for showing the costs is “X/Y”, where “X” is the cost of exploitation according to some exploitation cost scale and “Y” is the cost of remediation according to some remediation cost scale.
As there are two branches in FIG. 5 identified in step 1 of the algorithm, the implementation of the algorithm skips to step 3, where the critical branch must be identified. The cost of exploitation for the left-side branch is 2+4+7+1+6=20. The cost of exploitation for the right-side branch is 6+7+1+8=22. The branch having the lowest cost of exploitation is the left-side branch, and in accordance with step 3b of the algorithm the left-side branch is determined to be the critical branch. Within the left-side branch, the attacker step having the lowest cost of remediation is step 300 of applying method Z. Therefore, the algorithm decides that the highest priority attacker step to be blocked is step 300, as the method determined it to be the most cost-effective investment. It should be noted that even after blocking that attacker step, the networked system is still not fully protected—an attacker might still use the route of the right-side branch to compromise it, but his cost of exploitation would be somewhat higher after the remediation of blocking attacker step 300.
FIG. 6 provides another example of use of the exemplary algorithm. The paths of attack of FIG. 6 are nearly identical to those of FIG. 5, with the only difference being that in FIG. 6 step 408 (the third step of the right-side branch) “Use method S” is replaced with another copy of step 312, “Use method W”, where methods S and W of steps 408 and 312 have the same costs of exploitation and remediation. Note that in FIG. 6, step 312 of “Use method W” appears in both branches. The example of FIG. 6 may correspond to a case in which the CEO and the CFO share a common personal assistant. Applying the above algorithm to FIG. 6 results in the same recommendation provided in FIG. 5—the highest priority attacker step to be blocked is again step 300, the first step in the left-side branch, leaving an attacker the option to attack using the right-side branch.
However, a careful review of FIG. 6 indicates that blocking of attacker step 300 is clearly not the most cost-effective move. By blocking attacker step 312, “Use method W”, both branches can be blocked by a single remediation action. The cost of that single remediation action may be higher than the cost of blocking step 300 (in this example, 4 vs. 3), but the benefit gained from this slight additional cost is significant—instead of just somewhat increasing the cost of exploitation for the attacker while leaving paths of attack available to the attacker, blocking step 312 would completely block the attacker's ability to obtain his goal in compromising the networked system.
The flaw of the previous method, which causes it to recommend a non-optimal solution, is that it focuses on the “branches of attack” (i.e. on complete paths of attack) as fundamental and independent building blocks, ignoring connections or relations existing between different branches at a lower level.
There is thus a need in the art to have a method for prioritizing attacker steps to be blocked that provides cost-effective recommendations even when there are dependencies between different paths of attack.